Sentence Alignment Method Based on Maximum Entropy Model Using Anchor Sentences

The paper proposes a sentence alignment method based on maximum entropy model using anchor sentences to align ancient and modern Chinese sentences in historical classics. The method selects the sentence pairs with the same phrases at the beginning or the end of the sentence or with the same time phrases as anchor sentence pairs, which are employed to divide the paragraph into several sections. Then, the sentences in each section are aligned using dynamic programming algorithm according to the entropy calculated by maximum entropy model. The maximum entropy model employs improved Chinese co-occurrence character feature, length feature and sentence alignment mode feature. The Chinese cooccurrence characters feature is improved by giving different weights to characters in different position based on the contribution to align sentences. In the experiment performed on ShiJi, the precision and recall of the proposed method reaches 95.9% and 95.6% respectively, which outperforms other sentence alignment methods significantly.